Optimize metrics aggregate/persistent worker: seperate OAL and MAL workers and consume pools, and make the consumer thread sleep when no data in.#13414
Merged
wankai123 merged 5 commits intoapache:masterfrom Aug 14, 2025
Conversation
…` workers and consume pools, and make the consumer thread sleep when no data in.
wankai123
added
backend
There was a problem hiding this comment.
Pull Request Overview
This PR optimizes the metrics aggregation and persistence processing by separating OAL (Observability Analysis Language) and MAL (Meter Analysis Language) workers into dedicated thread pools with different configurations. The key optimization introduces a notification-based consumption pattern for MAL workers to reduce unnecessary resource usage when no data is available.
- Separates OAL and MAL workers into dedicated worker classes with optimized configurations
- Introduces notification-based consumer thread sleeping for MAL workers to reduce CPU usage
- Updates the data carrier consumer pool to support notifiable consumption patterns
Reviewed Changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| ConsumerPoolFactoryTest.java | Updates test to include new notifiable parameter in BulkConsumePool.Creator |
| MultipleChannelsConsumer.java | Adds notification-based consumption logic with sleep when no data |
| BulkConsumePool.java | Adds notifiable parameter and consumer notification capability |
| MetricsStreamProcessor.java | Implements factory pattern to create OAL/MAL specific workers |
| MetricsPersistentMinWorker.java | Converts to abstract base class with configurable pool parameters |
| MetricsPersistentMinOALWorker.java | OAL-specific persistent worker with optimized settings |
| MetricsPersistentMinMALWorker.java | MAL-specific persistent worker with notification support |
| MetricsAggregateWorker.java | Converts to abstract base class with configurable pool parameters |
| MetricsAggregateOALWorker.java | OAL-specific aggregate worker with high-throughput settings |
| MetricsAggregateMALWorker.java | MAL-specific aggregate worker with notification support |
| changes.md | Documents the optimization changes |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.
...in/java/org/apache/skywalking/oap/server/core/analysis/worker/MetricsAggregateOALWorker.java
Show resolved
Hide resolved
...in/java/org/apache/skywalking/oap/server/core/analysis/worker/MetricsAggregateMALWorker.java
Outdated
Show resolved
Hide resolved
...ava/org/apache/skywalking/oap/server/core/analysis/worker/MetricsPersistentMinOALWorker.java
Outdated
Show resolved
Hide resolved
...ava/org/apache/skywalking/oap/server/core/analysis/worker/MetricsPersistentMinMALWorker.java
Outdated
Show resolved
Hide resolved
wu-sheng
reviewed
Aug 14, 2025
wu-sheng
reviewed
Aug 14, 2025
...in/java/org/apache/skywalking/oap/server/core/analysis/worker/MetricsAggregateMALWorker.java
Outdated
Show resolved
Hide resolved
wu-sheng
reviewed
Aug 14, 2025
.../org/apache/skywalking/oap/server/library/datacarrier/consumer/MultipleChannelsConsumer.java
Outdated
Show resolved
Hide resolved
wu-sheng
previously approved these changes
Aug 14, 2025
wu-sheng
reviewed
Aug 14, 2025
wu-sheng
approved these changes
Aug 14, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CHANGESlog.Optimize metrics aggregate/persistent worker: seperate OAL and MAL workers and consume pools, and make the consumer thread sleep when no data in.